PART II: German Credit Score Classification Model Explainability

By: Krishna J

Importing necessary libraries

In [1]:
import pandas as pd
import numpy as np
import seaborn               as sns
import matplotlib.pyplot     as plt
import shap
import eli5
from sklearn.model_selection import train_test_split
#from sklearn.ensemble        import RandomForestClassifier
#from sklearn.linear_model    import LogisticRegression
from sklearn.preprocessing   import MinMaxScaler, StandardScaler
from sklearn.base            import TransformerMixin
from sklearn.pipeline        import Pipeline, FeatureUnion
from typing                  import List, Union, Dict
# Warnings will be used to silence various model warnings for tidier output
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline 
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
np.random.seed(0)
pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.

Importing the source dataset

In [2]:
german_xai=pd.read_csv('C:/Users/krish/Downloads/German-mapped.csv')

Converting categorical fields to numerical fields

In [3]:
german_xai=pd.get_dummies(german_xai,columns=['CreditHistory','Purpose','Savings','EmployDuration','Debtors','Collateral','OtherPayBackPlan','Property','Job'])
german_xai.head()
Out[3]:
NumMonths CreditAmount PayBackPercent Gender ResidenceDuration Age ExistingCredit Dependents Telephone Foreignworker ... OtherPayBackPlan_bank OtherPayBackPlan_none OtherPayBackPlan_stores Property_free Property_own Property_rent Job_management/self-emp/officer/highly qualif emp Job_skilled employee Job_unemp/unskilled-non resident Job_unskilled-resident
0 6 1169 4 1 4 1 2 1 1 1 ... 0 1 0 0 1 0 0 1 0 0
1 48 5951 2 0 2 0 1 1 0 1 ... 0 1 0 0 1 0 0 1 0 0
2 12 2096 2 1 3 1 1 2 0 1 ... 0 1 0 0 1 0 0 0 0 1
3 42 7882 2 1 4 1 1 2 0 1 ... 0 1 0 1 0 0 0 1 0 0
4 24 4870 3 1 4 1 2 2 0 1 ... 0 1 0 1 0 0 0 1 0 0

5 rows × 50 columns

In [4]:
german_xai.columns
Out[4]:
Index(['NumMonths', 'CreditAmount', 'PayBackPercent', 'Gender',
       'ResidenceDuration', 'Age', 'ExistingCredit', 'Dependents', 'Telephone',
       'Foreignworker', 'Marital_Status', 'CreditStatus',
       'CreditHistory_Delay', 'CreditHistory_none/paid', 'CreditHistory_other',
       'Purpose_CarNew', 'Purpose_CarUsed', 'Purpose_biz',
       'Purpose_domestic app', 'Purpose_education', 'Purpose_furniture/equip',
       'Purpose_others', 'Purpose_radio/tv', 'Purpose_repairs',
       'Purpose_retraining', 'Savings_500+', 'Savings_<500', 'Savings_none',
       'EmployDuration_1-4 yr', 'EmployDuration_4-7 yr',
       'EmployDuration_<1 yr', 'EmployDuration_>=7 yr',
       'EmployDuration_unemployed', 'Debtors_co-applicant',
       'Debtors_guarantor', 'Debtors_none', 'Collateral_car/other',
       'Collateral_real estate', 'Collateral_savings/life insurance',
       'Collateral_unknown/none', 'OtherPayBackPlan_bank',
       'OtherPayBackPlan_none', 'OtherPayBackPlan_stores', 'Property_free',
       'Property_own', 'Property_rent',
       'Job_management/self-emp/officer/highly qualif emp',
       'Job_skilled employee', 'Job_unemp/unskilled-non resident',
       'Job_unskilled-resident'],
      dtype='object')
In [5]:
german_xai = german_xai.reindex(columns=['NumMonths', 'CreditAmount', 'PayBackPercent', 'Gender',
       'ResidenceDuration', 'Age', 'ExistingCredit', 'Dependents', 'Telephone',
       'Foreignworker', 'Marital_Status',
       'CreditHistory_Delay', 'CreditHistory_none/paid', 'CreditHistory_other',
       'Purpose_CarNew', 'Purpose_CarUsed', 'Purpose_biz',
       'Purpose_domestic app', 'Purpose_education', 'Purpose_furniture/equip',
       'Purpose_others', 'Purpose_radio/tv', 'Purpose_repairs',
       'Purpose_retraining', 'Savings_500+', 'Savings_<500', 'Savings_none',
       'EmployDuration_1-4 yr', 'EmployDuration_4-7 yr',
       'EmployDuration_<1 yr', 'EmployDuration_>=7 yr',
       'EmployDuration_unemployed', 'Debtors_co-applicant',
       'Debtors_guarantor', 'Debtors_none', 'Collateral_car/other',
       'Collateral_real estate', 'Collateral_savings/life insurance',
       'Collateral_unknown/none', 'OtherPayBackPlan_bank',
       'OtherPayBackPlan_none', 'OtherPayBackPlan_stores', 'Property_free',
       'Property_own', 'Property_rent',
       'Job_management/self-emp/officer/highly qualif emp',
       'Job_skilled employee', 'Job_unemp/unskilled-non resident',
       'Job_unskilled-resident', 'CreditStatus'])
german_xai.head()
Out[5]:
NumMonths CreditAmount PayBackPercent Gender ResidenceDuration Age ExistingCredit Dependents Telephone Foreignworker ... OtherPayBackPlan_none OtherPayBackPlan_stores Property_free Property_own Property_rent Job_management/self-emp/officer/highly qualif emp Job_skilled employee Job_unemp/unskilled-non resident Job_unskilled-resident CreditStatus
0 6 1169 4 1 4 1 2 1 1 1 ... 1 0 0 1 0 0 1 0 0 1
1 48 5951 2 0 2 0 1 1 0 1 ... 1 0 0 1 0 0 1 0 0 0
2 12 2096 2 1 3 1 1 2 0 1 ... 1 0 0 1 0 0 0 0 1 1
3 42 7882 2 1 4 1 1 2 0 1 ... 1 0 1 0 0 0 1 0 0 1
4 24 4870 3 1 4 1 2 2 0 1 ... 1 0 1 0 0 0 1 0 0 0

5 rows × 50 columns

Writing data to csv file

In [6]:
german_xai.to_csv('C:/Users/krish/Downloads/German-encoded.csv', index=False)

Splitting into train and test data

In [7]:
X = german_xai.iloc[:, :-1]
y = german_xai['CreditStatus']
X.head()
y.head()
X_train,X_test,y_train, y_test = train_test_split(X,y, test_size=0.2, random_state=40,stratify=y)
Out[7]:
NumMonths CreditAmount PayBackPercent Gender ResidenceDuration Age ExistingCredit Dependents Telephone Foreignworker ... OtherPayBackPlan_bank OtherPayBackPlan_none OtherPayBackPlan_stores Property_free Property_own Property_rent Job_management/self-emp/officer/highly qualif emp Job_skilled employee Job_unemp/unskilled-non resident Job_unskilled-resident
0 6 1169 4 1 4 1 2 1 1 1 ... 0 1 0 0 1 0 0 1 0 0
1 48 5951 2 0 2 0 1 1 0 1 ... 0 1 0 0 1 0 0 1 0 0
2 12 2096 2 1 3 1 1 2 0 1 ... 0 1 0 0 1 0 0 0 0 1
3 42 7882 2 1 4 1 1 2 0 1 ... 0 1 0 1 0 0 0 1 0 0
4 24 4870 3 1 4 1 2 2 0 1 ... 0 1 0 1 0 0 0 1 0 0

5 rows × 49 columns

Out[7]:
0    1
1    0
2    1
3    1
4    0
Name: CreditStatus, dtype: int64
In [8]:
german_xai.dtypes
Out[8]:
NumMonths                                            int64
CreditAmount                                         int64
PayBackPercent                                       int64
Gender                                               int64
ResidenceDuration                                    int64
Age                                                  int64
ExistingCredit                                       int64
Dependents                                           int64
Telephone                                            int64
Foreignworker                                        int64
Marital_Status                                       int64
CreditHistory_Delay                                  uint8
CreditHistory_none/paid                              uint8
CreditHistory_other                                  uint8
Purpose_CarNew                                       uint8
Purpose_CarUsed                                      uint8
Purpose_biz                                          uint8
Purpose_domestic app                                 uint8
Purpose_education                                    uint8
Purpose_furniture/equip                              uint8
Purpose_others                                       uint8
Purpose_radio/tv                                     uint8
Purpose_repairs                                      uint8
Purpose_retraining                                   uint8
Savings_500+                                         uint8
Savings_<500                                         uint8
Savings_none                                         uint8
EmployDuration_1-4 yr                                uint8
EmployDuration_4-7 yr                                uint8
EmployDuration_<1 yr                                 uint8
EmployDuration_>=7 yr                                uint8
EmployDuration_unemployed                            uint8
Debtors_co-applicant                                 uint8
Debtors_guarantor                                    uint8
Debtors_none                                         uint8
Collateral_car/other                                 uint8
Collateral_real estate                               uint8
Collateral_savings/life insurance                    uint8
Collateral_unknown/none                              uint8
OtherPayBackPlan_bank                                uint8
OtherPayBackPlan_none                                uint8
OtherPayBackPlan_stores                              uint8
Property_free                                        uint8
Property_own                                         uint8
Property_rent                                        uint8
Job_management/self-emp/officer/highly qualif emp    uint8
Job_skilled employee                                 uint8
Job_unemp/unskilled-non resident                     uint8
Job_unskilled-resident                               uint8
CreditStatus                                         int64
dtype: object
In [9]:
import klib
klib.missingval_plot(X)
klib.missingval_plot(y)
No missing values found in the dataset.
No missing values found in the dataset.

Feature Selection

1. Using Mutual info classif

In [10]:
from sklearn.feature_selection import mutual_info_classif
mutual_info=mutual_info_classif(X_train, y_train,random_state=40)
mutual_info
Out[10]:
array([0.06019707, 0.02108839, 0.00223861, 0.        , 0.        ,
       0.        , 0.        , 0.        , 0.00353083, 0.02491331,
       0.        , 0.        , 0.00457068, 0.00844984, 0.        ,
       0.        , 0.01332007, 0.        , 0.        , 0.01102085,
       0.02957691, 0.        , 0.        , 0.00160791, 0.03429891,
       0.0457149 , 0.        , 0.        , 0.00819407, 0.00311788,
       0.02706385, 0.        , 0.        , 0.00205918, 0.0261869 ,
       0.        , 0.00280856, 0.00092909, 0.03215574, 0.02567867,
       0.00608715, 0.00376107, 0.        , 0.00802985, 0.        ,
       0.        , 0.        , 0.        , 0.        ])

Estimate mutual information for a discrete target variable.

Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency.

In [11]:
X.columns
Out[11]:
Index(['NumMonths', 'CreditAmount', 'PayBackPercent', 'Gender',
       'ResidenceDuration', 'Age', 'ExistingCredit', 'Dependents', 'Telephone',
       'Foreignworker', 'Marital_Status', 'CreditHistory_Delay',
       'CreditHistory_none/paid', 'CreditHistory_other', 'Purpose_CarNew',
       'Purpose_CarUsed', 'Purpose_biz', 'Purpose_domestic app',
       'Purpose_education', 'Purpose_furniture/equip', 'Purpose_others',
       'Purpose_radio/tv', 'Purpose_repairs', 'Purpose_retraining',
       'Savings_500+', 'Savings_<500', 'Savings_none', 'EmployDuration_1-4 yr',
       'EmployDuration_4-7 yr', 'EmployDuration_<1 yr',
       'EmployDuration_>=7 yr', 'EmployDuration_unemployed',
       'Debtors_co-applicant', 'Debtors_guarantor', 'Debtors_none',
       'Collateral_car/other', 'Collateral_real estate',
       'Collateral_savings/life insurance', 'Collateral_unknown/none',
       'OtherPayBackPlan_bank', 'OtherPayBackPlan_none',
       'OtherPayBackPlan_stores', 'Property_free', 'Property_own',
       'Property_rent', 'Job_management/self-emp/officer/highly qualif emp',
       'Job_skilled employee', 'Job_unemp/unskilled-non resident',
       'Job_unskilled-resident'],
      dtype='object')
In [12]:
mutual_info=pd.Series(mutual_info)
mutual_info.index=X_train.columns
mutual_info.sort_values(ascending=False)
Out[12]:
NumMonths                                            0.060197
Savings_<500                                         0.045715
Savings_500+                                         0.034299
Collateral_unknown/none                              0.032156
Purpose_others                                       0.029577
EmployDuration_>=7 yr                                0.027064
Debtors_none                                         0.026187
OtherPayBackPlan_bank                                0.025679
Foreignworker                                        0.024913
CreditAmount                                         0.021088
Purpose_biz                                          0.013320
Purpose_furniture/equip                              0.011021
CreditHistory_other                                  0.008450
EmployDuration_4-7 yr                                0.008194
Property_own                                         0.008030
OtherPayBackPlan_none                                0.006087
CreditHistory_none/paid                              0.004571
OtherPayBackPlan_stores                              0.003761
Telephone                                            0.003531
EmployDuration_<1 yr                                 0.003118
Collateral_real estate                               0.002809
PayBackPercent                                       0.002239
Debtors_guarantor                                    0.002059
Purpose_retraining                                   0.001608
Collateral_savings/life insurance                    0.000929
Property_free                                        0.000000
Property_rent                                        0.000000
Job_management/self-emp/officer/highly qualif emp    0.000000
Collateral_car/other                                 0.000000
Job_skilled employee                                 0.000000
Job_unemp/unskilled-non resident                     0.000000
Gender                                               0.000000
Savings_none                                         0.000000
Debtors_co-applicant                                 0.000000
EmployDuration_unemployed                            0.000000
EmployDuration_1-4 yr                                0.000000
ResidenceDuration                                    0.000000
Purpose_repairs                                      0.000000
Purpose_radio/tv                                     0.000000
Purpose_education                                    0.000000
Purpose_domestic app                                 0.000000
Purpose_CarUsed                                      0.000000
Purpose_CarNew                                       0.000000
CreditHistory_Delay                                  0.000000
Marital_Status                                       0.000000
Dependents                                           0.000000
ExistingCredit                                       0.000000
Age                                                  0.000000
Job_unskilled-resident                               0.000000
dtype: float64
In [13]:
mutual_info.sort_values(ascending=False).plot.bar(figsize=(15,5))
Out[13]:
<matplotlib.axes._subplots.AxesSubplot at 0x26dba9dd448>

Selecting top 10 features having highest dependencies w.r.to target variable CreditStatus

In [14]:
mutual_info.sort_values(ascending=False)[0:10]
Out[14]:
NumMonths                  0.060197
Savings_<500               0.045715
Savings_500+               0.034299
Collateral_unknown/none    0.032156
Purpose_others             0.029577
EmployDuration_>=7 yr      0.027064
Debtors_none               0.026187
OtherPayBackPlan_bank      0.025679
Foreignworker              0.024913
CreditAmount               0.021088
dtype: float64
In [15]:
german_xai_imp=german_xai[['Gender','Age','Marital_Status','NumMonths','Savings_<500','Savings_500+','Collateral_unknown/none','Purpose_others',
'EmployDuration_>=7 yr','Debtors_none','OtherPayBackPlan_bank','Foreignworker','CreditAmount','CreditStatus']]
german_xai_imp.head()
Out[15]:
Gender Age Marital_Status NumMonths Savings_<500 Savings_500+ Collateral_unknown/none Purpose_others EmployDuration_>=7 yr Debtors_none OtherPayBackPlan_bank Foreignworker CreditAmount CreditStatus
0 1 1 1 6 0 0 0 0 1 1 0 1 1169 1
1 0 0 0 48 1 0 0 0 0 1 0 1 5951 0
2 1 1 1 12 1 0 0 0 0 1 0 1 2096 1
3 1 1 1 42 1 0 0 0 0 0 0 1 7882 1
4 1 1 1 24 1 0 1 0 0 1 0 1 4870 0

2. Using correlation

In [16]:
corrMatrix = round(german_xai_imp.corr(),1)
corrMatrix
Out[16]:
Gender Age Marital_Status NumMonths Savings_<500 Savings_500+ Collateral_unknown/none Purpose_others EmployDuration_>=7 yr Debtors_none OtherPayBackPlan_bank Foreignworker CreditAmount CreditStatus
Gender 1.0 0.3 0.7 0.1 -0.0 -0.0 0.1 0.0 0.2 -0.0 0.0 -0.1 0.1 0.1
Age 0.3 1.0 0.2 0.0 -0.1 0.0 0.1 0.1 0.2 0.0 0.0 -0.1 0.0 0.1
Marital_Status 0.7 0.2 1.0 0.1 -0.1 -0.0 0.2 0.0 0.2 -0.0 0.0 -0.0 0.2 0.1
NumMonths 0.1 0.0 0.1 1.0 -0.0 -0.1 0.2 0.1 0.0 0.0 0.0 0.1 0.6 -0.2
Savings_<500 -0.0 -0.1 -0.1 -0.0 1.0 -0.5 -0.0 0.0 -0.1 -0.1 -0.0 0.0 -0.0 -0.2
Savings_500+ -0.0 0.0 -0.0 -0.1 -0.5 1.0 -0.0 -0.0 0.0 0.1 -0.0 0.0 -0.1 0.1
Collateral_unknown/none 0.1 0.1 0.2 0.2 -0.0 -0.0 1.0 0.1 0.2 0.0 0.1 0.1 0.2 -0.1
Purpose_others 0.0 0.1 0.0 0.1 0.0 -0.0 0.1 1.0 0.0 -0.1 0.1 -0.0 0.2 -0.0
EmployDuration_>=7 yr 0.2 0.2 0.2 0.0 -0.1 0.0 0.2 0.0 1.0 0.0 0.1 0.1 -0.0 0.1
Debtors_none -0.0 0.0 -0.0 0.0 -0.1 0.1 0.0 -0.1 0.0 1.0 -0.1 0.1 -0.0 0.0
OtherPayBackPlan_bank 0.0 0.0 0.0 0.0 -0.0 -0.0 0.1 0.1 0.1 -0.1 1.0 0.0 0.0 -0.1
Foreignworker -0.1 -0.1 -0.0 0.1 0.0 0.0 0.1 -0.0 0.1 0.1 0.0 1.0 0.1 -0.1
CreditAmount 0.1 0.0 0.2 0.6 -0.0 -0.1 0.2 0.2 -0.0 -0.0 0.0 0.1 1.0 -0.2
CreditStatus 0.1 0.1 0.1 -0.2 -0.2 0.1 -0.1 -0.0 0.1 0.0 -0.1 -0.1 -0.2 1.0
In [17]:
klib.corr_plot(german_xai_imp,annot=False)
Out[17]:
<matplotlib.axes._subplots.AxesSubplot at 0x26dbad780c8>
In [18]:
klib.corr_plot(german_xai_imp,target='CreditStatus')
Out[18]:
<matplotlib.axes._subplots.AxesSubplot at 0x26dbac4e1c8>

No higher correlation is observed between input variables (except gender, marital status (0.7) and credit amount, num of months (0.6) and between target variable and input variables. But since we are trying to understand the impact of protected variables let us retain them without dropping.

writing data to csv file

In [19]:
german_xai_imp.to_csv('C:/Users/krish/Downloads/German-reduced.csv', index=False)
In [20]:
#from sklearn.feature_selection import SelectPercentile
#selected_top=SelectPercentile(score_func=mutual_info_classif,  percentile=20)
#from sklearn.feature_selection import SelectKBest
#selected_top=SelectKBest(mutual_info_classif,k=10)
#selected_top.fit_transform(X_train,y_train)
In [21]:
#selected_top.fit_transform(X_train,y_train)
In [22]:
#X_sig=X_train.columns[selected_top.get_support()]
In [23]:
#X_sig
In [24]:
#X_train_sig=pd.DataFrame(X_train,columns=X_sig)
#X_test_sig=pd.DataFrame(X_test,columns=X_sig)
#X_train_sig.head()
#X_train_sig.shape
#X_test_sig.head()
#X_test_sig.shape

List of protected attributes

(https://arxiv.org/pdf/1811.11154.pdf)

In [25]:
from IPython.display  import Image
Image(filename='C:/Users/krish/Desktop/list of protected variables.png',width=500,height=30)
Out[25]:

From the above, we have 3 protected fields in our dataset:

1. Gender
2. Age
3. Marital Status

Now, let us identify previlege class in each protected attribute.

1.Gender

In [26]:
print(german_xai_imp['Gender'].value_counts())
german_xai_imp.groupby(['Gender'])['CreditStatus'].mean()
#https://arxiv.org/pdf/1810.01943.pdf, https://arxiv.org/pdf/2005.12379.pdf
1    690
0    310
Name: Gender, dtype: int64
Out[26]:
Gender
0    0.648387
1    0.723188
Name: CreditStatus, dtype: float64

Males(1) are more than females and for males(1) target variable CreditScore is more favorable having higher value for given number of males than female group average. Hence male(1) is privelieged class.

2.Age

In [27]:
print(german_xai_imp['Age'].value_counts())
german_xai_imp.groupby(['Age'])['CreditStatus'].mean()
1    810
0    190
Name: Age, dtype: int64
Out[27]:
Age
0    0.578947
1    0.728395
Name: CreditStatus, dtype: float64

Age >26: 1; else 0; so ppl above 26 are more and group average of ppl with age >26 is higher than the group of age < 26 ,so age(1) is priveleiged group

3. Marital Status

In [28]:
print(german_xai_imp['Marital_Status'].value_counts())
german_xai_imp.groupby(['Marital_Status'])['CreditStatus'].mean()
1    548
0    452
Name: Marital_Status, dtype: int64
Out[28]:
Marital_Status
0    0.659292
1    0.733577
Name: CreditStatus, dtype: float64

Singles(1) are more than not singles and for singles(1) target variable CreditScore is more favorable having higher value for given number of singles than non singles group average. Hence singles(1) is privelieged group

Converting Dataframe to aif compatible format

BinaryLabelDataset: Base class for all structured datasets with binary labels.

In [29]:
# Fairness metrics
from aif360.metrics import BinaryLabelDatasetMetric
from aif360.explainers import MetricTextExplainer
from aif360.metrics import ClassificationMetric
# Get DF into IBM format
from aif360 import datasets
aif_train_dataset = datasets.BinaryLabelDataset(favorable_label = 1, unfavorable_label = 0, df=german_xai_imp,
                                                      label_names=["CreditStatus"],
                                                     protected_attribute_names=["Age","Gender","Marital_Status"],
                                              privileged_protected_attributes = [1,1,1])
dataset_orig_train, dataset_orig_test = aif_train_dataset.split([0.7], shuffle=True)
In [30]:
dataset_orig_train.feature_names
Out[30]:
['Gender',
 'Age',
 'Marital_Status',
 'NumMonths',
 'Savings_<500',
 'Savings_500+',
 'Collateral_unknown/none',
 'Purpose_others',
 'EmployDuration_>=7 yr',
 'Debtors_none',
 'OtherPayBackPlan_bank',
 'Foreignworker',
 'CreditAmount']

Measuring fairness:

Disparate Impact

a) With respect to Gender

In [31]:
# Disparate impact measurement for gender
metric_aif_train_ready_gender = BinaryLabelDatasetMetric(
        aif_train_dataset,
        unprivileged_groups=[{"Age":0,"Gender":0,"Marital_Status":0}],  privileged_groups=[{"Age":1,"Gender":1,"Marital_Status":1}]) 
explainer_aif_train_ready_gender = MetricTextExplainer(metric_aif_train_ready_gender)

print(explainer_aif_train_ready_gender.disparate_impact())
print("Difference in mean outcomes between unprivileged and privileged groups of gender = %f" % metric_aif_train_ready_gender.mean_difference())
Disparate impact (probability of favorable outcome for unprivileged instances / probability of favorable outcome for privileged instances): 0.7385093167701864
Difference in mean outcomes between unprivileged and privileged groups of gender = -0.195587

Handling bias: Reweighing

In [32]:
from aif360.algorithms.preprocessing import Reweighing
privileged_groups = [{'Gender': 1}]
unprivileged_groups = [{'Gender': 0}]
RW_gender = Reweighing(unprivileged_groups=unprivileged_groups,
                privileged_groups=privileged_groups)
dataset_aif_tranf_gender = RW_gender.fit_transform(dataset_orig_train)
metric_transf_train_gender = BinaryLabelDatasetMetric(dataset_aif_tranf_gender, 
                                               unprivileged_groups=unprivileged_groups,
                                               privileged_groups=privileged_groups)

print("Difference in mean outcomes between unprivileged and privileged groups of gender = %f" % metric_transf_train_gender.mean_difference())
WARNING:root:No module named 'numba.decorators': LFR will be unavailable. To install, run:
pip install 'aif360[LFR]'
Difference in mean outcomes between unprivileged and privileged groups of gender = 0.000000

b) with respect to Age

In [33]:
# Disparate impact measurement for age
metric_aif_train_ready_age = BinaryLabelDatasetMetric(
        aif_train_dataset,
        unprivileged_groups=[{"Age":0}],
        privileged_groups=[{"Age":1}])
explainer_aif_train_ready_age = MetricTextExplainer(metric_aif_train_ready_age)

print(explainer_aif_train_ready_age.disparate_impact())
print("Difference in mean outcomes between unprivileged and privileged groups of age = %f" % metric_aif_train_ready_age.mean_difference())
Disparate impact (probability of favorable outcome for unprivileged instances / probability of favorable outcome for privileged instances): 0.7948260481712757
Difference in mean outcomes between unprivileged and privileged groups of age = -0.149448

Handling bias: Reweighing

In [34]:
from aif360.algorithms.preprocessing import Reweighing
privileged_groups = [{'Age': 1}]
unprivileged_groups = [{'Age': 0}]
RW_age = Reweighing(unprivileged_groups=unprivileged_groups,
                privileged_groups=privileged_groups)
dataset_aif_tranf_age = RW_age.fit_transform(dataset_orig_train)
metric_transf_train_age = BinaryLabelDatasetMetric(dataset_aif_tranf_age, 
                                               unprivileged_groups=unprivileged_groups,
                                               privileged_groups=privileged_groups)

print("Difference in mean outcomes between unprivileged and privileged groups of age = %f" % metric_transf_train_age.mean_difference())
Difference in mean outcomes between unprivileged and privileged groups of age = -0.000000
In [35]:
metric_transf_train_age
Out[35]:
<aif360.metrics.binary_label_dataset_metric.BinaryLabelDatasetMetric at 0x26dbc0842c8>

c) with respect to Marital Status

In [36]:
# Disparate impact measurement for age
metric_aif_train_ready_marital = BinaryLabelDatasetMetric(
        aif_train_dataset,
        unprivileged_groups=[{"Marital_Status":0}],
        privileged_groups=[{"Marital_Status":1}])
explainer_aif_train_ready_marital = MetricTextExplainer(metric_aif_train_ready_marital)

print(explainer_aif_train_ready_marital.disparate_impact())
print("Difference in mean outcomes between unprivileged and privileged groups of marital status = %f" % metric_aif_train_ready_marital.mean_difference())
Disparate impact (probability of favorable outcome for unprivileged instances / probability of favorable outcome for privileged instances): 0.8987364064632589
Difference in mean outcomes between unprivileged and privileged groups of marital status = -0.074285

Handling bias: Reweighing

In [37]:
from aif360.algorithms.preprocessing import Reweighing
privileged_groups = [{'Marital_Status': 1}]
unprivileged_groups = [{'Marital_Status': 0}]
RW_Marital = Reweighing(unprivileged_groups=unprivileged_groups,
                privileged_groups=privileged_groups)
dataset_aif_tranf_marital = RW_Marital.fit_transform(dataset_orig_train)
metric_transf_train_marital = BinaryLabelDatasetMetric(dataset_aif_tranf_marital, 
                                               unprivileged_groups=unprivileged_groups,
                                               privileged_groups=privileged_groups)

print("Difference in mean outcomes between unprivileged and privileged groups of marital status = %f" % metric_transf_train_marital.mean_difference())
Difference in mean outcomes between unprivileged and privileged groups of marital status = -0.000000

Building a ML model

1.RANDOM FOREST

In [38]:
#Seting the Hyper Parameters
param_grid = {"max_depth": [3,5,7,None],
              "n_estimators":[3,5,10,15,20],
              "max_features": [4,7,15]}
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
#Creating the classifier
rf_model = RandomForestClassifier(random_state=40)
grid_search = GridSearchCV(rf_model, param_grid=param_grid, cv=5, scoring='recall', verbose=0)
model = grid_search

1.a) with age as protected variable in the dataset

In [39]:
mdl_age = model.fit(dataset_aif_tranf_age.features, dataset_aif_tranf_age.labels.ravel())
In [40]:
rf_shap_values = shap.KernelExplainer(grid_search.predict,dataset_aif_tranf_age.features)
WARNING:shap:Using 700 background data samples could cause slower run times. Consider using shap.sample(data, K) or shap.kmeans(data, K) to summarize the background as K samples.

1.a.1) Model feature importance

In [41]:
importances = model.best_estimator_.feature_importances_
indices = np.argsort(importances)
features = dataset_aif_tranf_age.feature_names
#https://stackoverflow.com/questions/48377296/get-feature-importance-from-gridsearchcv
In [42]:
importances

plt.figure(figsize=(20,30))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='b', align='center')
plt.yticks(range(len(indices)), [features[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
Out[42]:
array([0.03054944, 0.14122543, 0.04050242, 0.21946099, 0.15563724,
       0.01814715, 0.0132588 , 0.00077236, 0.02369723, 0.        ,
       0.03601172, 0.        , 0.32073723])
Out[42]:
<Figure size 1440x2160 with 0 Axes>
Out[42]:
Text(0.5, 1.0, 'Feature Importances')
Out[42]:
<BarContainer object of 13 artists>
Out[42]:
([<matplotlib.axis.YTick at 0x26dbc16c408>,
  <matplotlib.axis.YTick at 0x26dbc16ce88>,
  <matplotlib.axis.YTick at 0x26dbc093e48>,
  <matplotlib.axis.YTick at 0x26dbc61f5c8>,
  <matplotlib.axis.YTick at 0x26dbc623508>,
  <matplotlib.axis.YTick at 0x26dbc623dc8>,
  <matplotlib.axis.YTick at 0x26dbc624708>,
  <matplotlib.axis.YTick at 0x26dbc6290c8>,
  <matplotlib.axis.YTick at 0x26dbc6298c8>,
  <matplotlib.axis.YTick at 0x26dbc62c308>,
  <matplotlib.axis.YTick at 0x26dbc62cd88>,
  <matplotlib.axis.YTick at 0x26dbc629e48>,
  <matplotlib.axis.YTick at 0x26dbc6314c8>],
 <a list of 13 Text yticklabel objects>)
Out[42]:
Text(0.5, 0, 'Relative Importance')

Features that are important in the model are given above.

1.a.2) Model Explainability

1.a.2.a) Using SHAP

In [43]:
mdl_age.best_params_
type(model)
explainer = shap.TreeExplainer(grid_search.best_estimator_)
shap_values_a=explainer.shap_values(dataset_aif_tranf_age.features, dataset_aif_tranf_age.labels.ravel())
#https://github.com/slundberg/shap/issues/968
Out[43]:
{'max_depth': 3, 'max_features': 7, 'n_estimators': 10}
Out[43]:
sklearn.model_selection._search.GridSearchCV
In [44]:
shap_values_a
Out[44]:
[array([[-5.76477868e-03, -2.32884435e-02, -6.61527689e-03, ...,
         -5.01754168e-03,  0.00000000e+00,  7.49099651e-02],
        [-7.72110140e-03, -2.90583936e-02, -3.04146469e-03, ...,
         -1.62416087e-03,  0.00000000e+00, -5.68452118e-02],
        [-7.72110140e-03, -2.36312465e-02, -8.19963297e-03, ...,
         -1.75775342e-03,  0.00000000e+00, -2.22358858e-02],
        ...,
        [ 8.26887424e-03,  6.77228995e-02,  3.70893200e-03, ...,
          1.56349442e-02,  0.00000000e+00, -2.36211556e-02],
        [ 7.70202887e-05, -1.87248363e-02, -8.19963297e-03, ...,
         -5.53378520e-03,  0.00000000e+00,  9.27872387e-03],
        [ 2.04286235e-02, -2.13479386e-02,  4.29016770e-03, ...,
         -1.11661151e-03,  0.00000000e+00, -1.67289743e-02]]),
 array([[ 5.76477868e-03,  2.32884435e-02,  6.61527689e-03, ...,
          5.01754168e-03,  0.00000000e+00, -7.49099651e-02],
        [ 7.72110140e-03,  2.90583936e-02,  3.04146469e-03, ...,
          1.62416087e-03,  0.00000000e+00,  5.68452118e-02],
        [ 7.72110140e-03,  2.36312465e-02,  8.19963297e-03, ...,
          1.75775342e-03,  0.00000000e+00,  2.22358858e-02],
        ...,
        [-8.26887424e-03, -6.77228995e-02, -3.70893200e-03, ...,
         -1.56349442e-02,  0.00000000e+00,  2.36211556e-02],
        [-7.70202887e-05,  1.87248363e-02,  8.19963297e-03, ...,
          5.53378520e-03,  0.00000000e+00, -9.27872387e-03],
        [-2.04286235e-02,  2.13479386e-02, -4.29016770e-03, ...,
          1.11661151e-03,  0.00000000e+00,  1.67289743e-02]])]

The shap_values[0] are explanations with respect to the negative class, while shap_values[1] are explanations with respect to the positive class.

Features in blue pushes the base value towards lowest values and features in red moves base levels towards higher values.

In [45]:
shap.initjs()
shap.force_plot(explainer.expected_value[0],shap_values_a[0][0], dataset_aif_tranf_age.feature_names)
#https://github.com/slundberg/shap
#https://github.com/slundberg/shap/issues/279
Out[45]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [46]:
shap.initjs()
shap.force_plot(explainer.expected_value[1],shap_values_a[1][0], dataset_aif_tranf_age.feature_names)
Out[46]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [47]:
dataset_aif_tranf_age.feature_names
Out[47]:
['Gender',
 'Age',
 'Marital_Status',
 'NumMonths',
 'Savings_<500',
 'Savings_500+',
 'Collateral_unknown/none',
 'Purpose_others',
 'EmployDuration_>=7 yr',
 'Debtors_none',
 'OtherPayBackPlan_bank',
 'Foreignworker',
 'CreditAmount']
In [48]:
shap.force_plot(explainer.expected_value[0],
                shap_values_a[0][:,:], dataset_aif_tranf_age.features[:,:],feature_names = dataset_aif_tranf_age.feature_names)
Out[48]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [49]:
shap.force_plot(explainer.expected_value[1],
                shap_values_a[1][:,:], dataset_aif_tranf_age.features[:,:],feature_names = dataset_aif_tranf_age.feature_names)
Out[49]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [50]:
p = shap.summary_plot(shap_values_a, dataset_aif_tranf_age.features, feature_names=dataset_aif_tranf_age.feature_names) 
display(p)
None

Variables with higher impact are Age,CreditAmount,NumMonths,Savings etc

In [51]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value[0], shap_values_a[0][0],feature_names=dataset_aif_tranf_age.feature_names)

Interpretation of graph: https://shap.readthedocs.io/en/latest/example_notebooks/overviews/An%20introduction%20to%20explainable%20AI%20with%20Shapley%20values.html

The above explanation shows features each contributing to push the model output from the base value (the average model output over the training dataset we passed) to the model output. Features pushing the prediction higher are shown in red, those pushing the prediction lower are in blue.

f(x)- model output impacted by features; E(f(x))- expected output.

One the fundemental properties of Shapley values is that they always sum up to the difference between the game outcome when all players are present and the game outcome when no players are present. For machine learning models this means that SHAP values of all the input features will always sum up to the difference between baseline (expected) model output and the current model output for the prediction being explained.

In [52]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value[1], shap_values_a[1][0],feature_names=dataset_aif_tranf_age.feature_names)

1.a.2.b) Using eli5

In [53]:
#!pip install eli5
from eli5.sklearn import PermutationImportance
In [54]:
perm_age = PermutationImportance(model).fit(dataset_aif_tranf_age.features, dataset_aif_tranf_age.labels.ravel())
In [55]:
perm_imp_1=eli5.show_weights(perm_age,feature_names = dataset_aif_tranf_age.feature_names)
perm_imp_1
plt.show()
Out[55]:
Weight Feature
0.0233 ± 0.0080 NumMonths
0.0192 ± 0.0080 CreditAmount
0.0118 ± 0.0162 Age
0.0106 ± 0.0031 Savings_<500
0.0024 ± 0.0031 Marital_Status
0.0012 ± 0.0033 OtherPayBackPlan_bank
0.0012 ± 0.0020 Gender
0 ± 0.0000 Foreignworker
0 ± 0.0000 Debtors_none
0 ± 0.0000 EmployDuration_>=7 yr
0 ± 0.0000 Purpose_others
-0.0008 ± 0.0033 Collateral_unknown/none
-0.0012 ± 0.0033 Savings_500+

eli5 provides a way to compute feature importances for any black-box estimator by measuring how score decreases when a feature is not available; the method is also known as “permutation importance” or “Mean Decrease Accuracy (MDA)”.

The first number in each row shows how much model performance decreased with a random shuffling (in this case, using "accuracy" as the performance metric).

Like most things in data science, there is some randomness to the exact performance change from a shuffling a column. We measure the amount of randomness in our permutation importance calculation by repeating the process with multiple shuffles. The number after the ± measures how performance varied from one-reshuffling to the next.

You'll occasionally see negative values for permutation importances. In those cases, the predictions on the shuffled (or noisy) data happened to be more accurate than the real data. This happens when the feature didn't matter (should have had an importance close to 0), but random chance caused the predictions on shuffled data to be more accurate. This is more common with small datasets, like the one in this example, because there is more room for luck/chance.

https://www.kaggle.com/dansbecker/permutation-importance

1.b) with gender as protected variable in the dataset

In [56]:
mdl_gender = model.fit(dataset_aif_tranf_gender.features, dataset_aif_tranf_gender.labels.ravel())
In [57]:
rf_shap_values = shap.KernelExplainer(grid_search.predict,dataset_aif_tranf_gender.features)
WARNING:shap:Using 700 background data samples could cause slower run times. Consider using shap.sample(data, K) or shap.kmeans(data, K) to summarize the background as K samples.

1.b.1) Model feature importance

In [58]:
importances = model.best_estimator_.feature_importances_
indices = np.argsort(importances)
features = dataset_aif_tranf_gender.feature_names
#https://stackoverflow.com/questions/48377296/get-feature-importance-from-gridsearchcv
In [59]:
importances

plt.figure(figsize=(20,30))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='b', align='center')
plt.yticks(range(len(indices)), [features[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
Out[59]:
array([0.03054944, 0.14122543, 0.04050242, 0.21946099, 0.15563724,
       0.01814715, 0.0132588 , 0.00077236, 0.02369723, 0.        ,
       0.03601172, 0.        , 0.32073723])
Out[59]:
<Figure size 1440x2160 with 0 Axes>
Out[59]:
Text(0.5, 1.0, 'Feature Importances')
Out[59]:
<BarContainer object of 13 artists>
Out[59]:
([<matplotlib.axis.YTick at 0x26dbdc9a408>,
  <matplotlib.axis.YTick at 0x26dbddff448>,
  <matplotlib.axis.YTick at 0x26dbddf5348>,
  <matplotlib.axis.YTick at 0x26dbdc62388>,
  <matplotlib.axis.YTick at 0x26dbde09fc8>,
  <matplotlib.axis.YTick at 0x26dbde09c08>,
  <matplotlib.axis.YTick at 0x26dbdc68cc8>,
  <matplotlib.axis.YTick at 0x26dbdc68548>,
  <matplotlib.axis.YTick at 0x26dbde093c8>,
  <matplotlib.axis.YTick at 0x26dbdc6c0c8>,
  <matplotlib.axis.YTick at 0x26dbdc86788>,
  <matplotlib.axis.YTick at 0x26dbdc7b1c8>,
  <matplotlib.axis.YTick at 0x26dbdc7b108>],
 <a list of 13 Text yticklabel objects>)
Out[59]:
Text(0.5, 0, 'Relative Importance')

1.b.2) Model Explainability

1.b.2.a) Using SHAP

In [60]:
mdl_gender.best_params_
type(model)
explainer = shap.TreeExplainer(grid_search.best_estimator_)
shap_values_b=explainer.shap_values(dataset_aif_tranf_gender.features, dataset_aif_tranf_gender.labels.ravel())
#https://github.com/slundberg/shap/issues/968
Out[60]:
{'max_depth': 3, 'max_features': 7, 'n_estimators': 10}
Out[60]:
sklearn.model_selection._search.GridSearchCV
In [61]:
shap_values_b
Out[61]:
[array([[-5.76477868e-03, -2.32884435e-02, -6.61527689e-03, ...,
         -5.01754168e-03,  0.00000000e+00,  7.49099651e-02],
        [-7.72110140e-03, -2.90583936e-02, -3.04146469e-03, ...,
         -1.62416087e-03,  0.00000000e+00, -5.68452118e-02],
        [-7.72110140e-03, -2.36312465e-02, -8.19963297e-03, ...,
         -1.75775342e-03,  0.00000000e+00, -2.22358858e-02],
        ...,
        [ 8.26887424e-03,  6.77228995e-02,  3.70893200e-03, ...,
          1.56349442e-02,  0.00000000e+00, -2.36211556e-02],
        [ 7.70202887e-05, -1.87248363e-02, -8.19963297e-03, ...,
         -5.53378520e-03,  0.00000000e+00,  9.27872387e-03],
        [ 2.04286235e-02, -2.13479386e-02,  4.29016770e-03, ...,
         -1.11661151e-03,  0.00000000e+00, -1.67289743e-02]]),
 array([[ 5.76477868e-03,  2.32884435e-02,  6.61527689e-03, ...,
          5.01754168e-03,  0.00000000e+00, -7.49099651e-02],
        [ 7.72110140e-03,  2.90583936e-02,  3.04146469e-03, ...,
          1.62416087e-03,  0.00000000e+00,  5.68452118e-02],
        [ 7.72110140e-03,  2.36312465e-02,  8.19963297e-03, ...,
          1.75775342e-03,  0.00000000e+00,  2.22358858e-02],
        ...,
        [-8.26887424e-03, -6.77228995e-02, -3.70893200e-03, ...,
         -1.56349442e-02,  0.00000000e+00,  2.36211556e-02],
        [-7.70202887e-05,  1.87248363e-02,  8.19963297e-03, ...,
          5.53378520e-03,  0.00000000e+00, -9.27872387e-03],
        [-2.04286235e-02,  2.13479386e-02, -4.29016770e-03, ...,
          1.11661151e-03,  0.00000000e+00,  1.67289743e-02]])]
In [62]:
shap.initjs()
shap.force_plot(explainer.expected_value[0],shap_values_b[0][0], dataset_aif_tranf_gender.feature_names)
#https://github.com/slundberg/shap
#https://github.com/slundberg/shap/issues/279
Out[62]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.

The shap_values[0] are explanations with respect to the negative class, while shap_values[1] are explanations with respect to the positive class.

In [63]:
shap.initjs()
shap.force_plot(explainer.expected_value[1],shap_values_b[1][0], dataset_aif_tranf_gender.feature_names)
Out[63]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [64]:
dataset_aif_tranf_gender.feature_names
Out[64]:
['Gender',
 'Age',
 'Marital_Status',
 'NumMonths',
 'Savings_<500',
 'Savings_500+',
 'Collateral_unknown/none',
 'Purpose_others',
 'EmployDuration_>=7 yr',
 'Debtors_none',
 'OtherPayBackPlan_bank',
 'Foreignworker',
 'CreditAmount']
In [65]:
shap.force_plot(explainer.expected_value[0],
                shap_values_b[0][:,:], dataset_aif_tranf_gender.features[:,:],feature_names = dataset_aif_tranf_gender.feature_names)
Out[65]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [66]:
shap.force_plot(explainer.expected_value[1],
                shap_values_b[1][:,:], dataset_aif_tranf_gender.features[:,:],feature_names = dataset_aif_tranf_gender.feature_names)
Out[66]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [67]:
p = shap.summary_plot(shap_values_b, dataset_aif_tranf_gender.features, feature_names=dataset_aif_tranf_gender.feature_names) 
display(p)
None
In [68]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value[0], shap_values_b[0][0],feature_names=dataset_aif_tranf_gender.feature_names)

Interpretation of graph: https://shap.readthedocs.io/en/latest/example_notebooks/overviews/An%20introduction%20to%20explainable%20AI%20with%20Shapley%20values.html

f(x)- model output impacted by features; E(f(x))- expected output.

The above explanation shows features each contributing to push the model output from the base value (the average model output over the training dataset we passed) to the model output. Features pushing the prediction higher are shown in red, those pushing the prediction lower are in blue.

One the fundemental properties of Shapley values is that they always sum up to the difference between the game outcome when all players are present and the game outcome when no players are present. For machine learning models this means that SHAP values of all the input features will always sum up to the difference between baseline (expected) model output and the current model output for the prediction being explained.

In [69]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value[1], shap_values_b[1][0],feature_names=dataset_aif_tranf_gender.feature_names)

1.b.2.b) Using ELI5

In [70]:
#!pip install eli5
from eli5.sklearn import PermutationImportance
In [71]:
perm_gender = PermutationImportance(model).fit(dataset_aif_tranf_gender.features, dataset_aif_tranf_gender.labels.ravel())
In [72]:
perm_imp_2=eli5.show_weights(perm_gender,feature_names = dataset_aif_tranf_gender.feature_names)
perm_imp_2
plt.show()
Out[72]:
Weight Feature
0.0200 ± 0.0094 NumMonths
0.0155 ± 0.0084 CreditAmount
0.0127 ± 0.0031 Savings_<500
0.0090 ± 0.0099 Age
0.0020 ± 0.0000 Marital_Status
0.0012 ± 0.0033 OtherPayBackPlan_bank
0.0012 ± 0.0020 Gender
0 ± 0.0000 Foreignworker
0 ± 0.0000 Debtors_none
0 ± 0.0000 EmployDuration_>=7 yr
0 ± 0.0000 Purpose_others
-0.0004 ± 0.0031 Collateral_unknown/none
-0.0012 ± 0.0020 Savings_500+

eli5 provides a way to compute feature importances for any black-box estimator by measuring how score decreases when a feature is not available; the method is also known as “permutation importance” or “Mean Decrease Accuracy (MDA)”.

1.c) with marital status as protected variable in the dataset

In [73]:
mdl_marital = model.fit(dataset_aif_tranf_marital.features, dataset_aif_tranf_marital.labels.ravel())
In [74]:
rf_shap_values = shap.KernelExplainer(grid_search.predict,dataset_aif_tranf_marital.features)
WARNING:shap:Using 700 background data samples could cause slower run times. Consider using shap.sample(data, K) or shap.kmeans(data, K) to summarize the background as K samples.

1.c.1) Model feature importance

In [75]:
importances = model.best_estimator_.feature_importances_
indices = np.argsort(importances)
features = dataset_aif_tranf_marital.feature_names
#https://stackoverflow.com/questions/48377296/get-feature-importance-from-gridsearchcv
In [76]:
importances

plt.figure(figsize=(20,30))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='b', align='center')
plt.yticks(range(len(indices)), [features[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
Out[76]:
array([0.03054944, 0.14122543, 0.04050242, 0.21946099, 0.15563724,
       0.01814715, 0.0132588 , 0.00077236, 0.02369723, 0.        ,
       0.03601172, 0.        , 0.32073723])
Out[76]:
<Figure size 1440x2160 with 0 Axes>
Out[76]:
Text(0.5, 1.0, 'Feature Importances')
Out[76]:
<BarContainer object of 13 artists>
Out[76]:
([<matplotlib.axis.YTick at 0x26dc0466a08>,
  <matplotlib.axis.YTick at 0x26dc05d7048>,
  <matplotlib.axis.YTick at 0x26dc04567c8>,
  <matplotlib.axis.YTick at 0x26dc0496308>,
  <matplotlib.axis.YTick at 0x26dc0445348>,
  <matplotlib.axis.YTick at 0x26dc0445688>,
  <matplotlib.axis.YTick at 0x26dc04456c8>,
  <matplotlib.axis.YTick at 0x26dc045b608>,
  <matplotlib.axis.YTick at 0x26dc045b788>,
  <matplotlib.axis.YTick at 0x26dc044e388>,
  <matplotlib.axis.YTick at 0x26dc0453748>,
  <matplotlib.axis.YTick at 0x26dc0443d48>,
  <matplotlib.axis.YTick at 0x26dc0478888>],
 <a list of 13 Text yticklabel objects>)
Out[76]:
Text(0.5, 0, 'Relative Importance')

1.c.2) Model Explainability

1.c.2.a) Using SHAP

In [77]:
mdl_gender.best_params_
type(model)
explainer = shap.TreeExplainer(grid_search.best_estimator_)
shap_values_c=explainer.shap_values(dataset_aif_tranf_marital.features, dataset_aif_tranf_marital.labels.ravel())
#https://github.com/slundberg/shap/issues/968
Out[77]:
{'max_depth': 3, 'max_features': 7, 'n_estimators': 10}
Out[77]:
sklearn.model_selection._search.GridSearchCV
In [78]:
shap_values_c
Out[78]:
[array([[-5.76477868e-03, -2.32884435e-02, -6.61527689e-03, ...,
         -5.01754168e-03,  0.00000000e+00,  7.49099651e-02],
        [-7.72110140e-03, -2.90583936e-02, -3.04146469e-03, ...,
         -1.62416087e-03,  0.00000000e+00, -5.68452118e-02],
        [-7.72110140e-03, -2.36312465e-02, -8.19963297e-03, ...,
         -1.75775342e-03,  0.00000000e+00, -2.22358858e-02],
        ...,
        [ 8.26887424e-03,  6.77228995e-02,  3.70893200e-03, ...,
          1.56349442e-02,  0.00000000e+00, -2.36211556e-02],
        [ 7.70202887e-05, -1.87248363e-02, -8.19963297e-03, ...,
         -5.53378520e-03,  0.00000000e+00,  9.27872387e-03],
        [ 2.04286235e-02, -2.13479386e-02,  4.29016770e-03, ...,
         -1.11661151e-03,  0.00000000e+00, -1.67289743e-02]]),
 array([[ 5.76477868e-03,  2.32884435e-02,  6.61527689e-03, ...,
          5.01754168e-03,  0.00000000e+00, -7.49099651e-02],
        [ 7.72110140e-03,  2.90583936e-02,  3.04146469e-03, ...,
          1.62416087e-03,  0.00000000e+00,  5.68452118e-02],
        [ 7.72110140e-03,  2.36312465e-02,  8.19963297e-03, ...,
          1.75775342e-03,  0.00000000e+00,  2.22358858e-02],
        ...,
        [-8.26887424e-03, -6.77228995e-02, -3.70893200e-03, ...,
         -1.56349442e-02,  0.00000000e+00,  2.36211556e-02],
        [-7.70202887e-05,  1.87248363e-02,  8.19963297e-03, ...,
          5.53378520e-03,  0.00000000e+00, -9.27872387e-03],
        [-2.04286235e-02,  2.13479386e-02, -4.29016770e-03, ...,
          1.11661151e-03,  0.00000000e+00,  1.67289743e-02]])]
In [79]:
shap.initjs()
shap.force_plot(explainer.expected_value[0],shap_values_c[0][0], dataset_aif_tranf_marital.feature_names)
#https://github.com/slundberg/shap
#https://github.com/slundberg/shap/issues/279
Out[79]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.

The shap_values[0] are explanations with respect to the negative class, while shap_values[1] are explanations with respect to the positive class.

In [80]:
shap.initjs()
shap.force_plot(explainer.expected_value[1],shap_values_c[1][0], dataset_aif_tranf_marital.feature_names)
Out[80]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [81]:
dataset_aif_tranf_marital.feature_names
Out[81]:
['Gender',
 'Age',
 'Marital_Status',
 'NumMonths',
 'Savings_<500',
 'Savings_500+',
 'Collateral_unknown/none',
 'Purpose_others',
 'EmployDuration_>=7 yr',
 'Debtors_none',
 'OtherPayBackPlan_bank',
 'Foreignworker',
 'CreditAmount']
In [82]:
shap.force_plot(explainer.expected_value[0],
                shap_values_c[0][:,:], dataset_aif_tranf_marital.features[:,:],feature_names = dataset_aif_tranf_marital.feature_names)
Out[82]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [83]:
shap.force_plot(explainer.expected_value[1],
                shap_values_c[1][:,:], dataset_aif_tranf_marital.features[:,:],feature_names = dataset_aif_tranf_marital.feature_names)
Out[83]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [84]:
p = shap.summary_plot(shap_values_c, dataset_aif_tranf_marital.features, feature_names=dataset_aif_tranf_marital.feature_names) 
display(p)
None
In [85]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value[0], shap_values_c[0][0],feature_names=dataset_aif_tranf_marital.feature_names)

Interpretation of graph: https://shap.readthedocs.io/en/latest/example_notebooks/overviews/An%20introduction%20to%20explainable%20AI%20with%20Shapley%20values.html

f(x)- model output impacted by features; E(f(x))- expected output.

The above explanation shows features each contributing to push the model output from the base value (the average model output over the training dataset we passed) to the model output. Features pushing the prediction higher are shown in red, those pushing the prediction lower are in blue.

One the fundemental properties of Shapley values is that they always sum up to the difference between the game outcome when all players are present and the game outcome when no players are present. For machine learning models this means that SHAP values of all the input features will always sum up to the difference between baseline (expected) model output and the current model output for the prediction being explained.

In [86]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value[1], shap_values_c[1][0],feature_names=dataset_aif_tranf_marital.feature_names)

1.c.2.a) Using ELI5

In [87]:
#!pip install eli5
from eli5.sklearn import PermutationImportance
In [88]:
perm_marital = PermutationImportance(model).fit(dataset_aif_tranf_marital.features, dataset_aif_tranf_marital.labels.ravel())
In [89]:
perm_imp_3=eli5.show_weights(perm_marital,feature_names = dataset_aif_tranf_marital.feature_names)
perm_imp_3
plt.show()
Out[89]:
Weight Feature
0.0220 ± 0.0060 NumMonths
0.0151 ± 0.0095 CreditAmount
0.0118 ± 0.0060 Age
0.0090 ± 0.0055 Savings_<500
0.0024 ± 0.0031 Marital_Status
0.0016 ± 0.0031 OtherPayBackPlan_bank
0.0016 ± 0.0031 Gender
0 ± 0.0000 Foreignworker
0 ± 0.0000 Debtors_none
0 ± 0.0000 EmployDuration_>=7 yr
0 ± 0.0000 Purpose_others
0 ± 0.0000 Savings_500+
-0.0016 ± 0.0016 Collateral_unknown/none

eli5 provides a way to compute feature importances for any black-box estimator by measuring how score decreases when a feature is not available; the method is also known as “permutation importance” or “Mean Decrease Accuracy (MDA)”.

2. XGBOOST

In [90]:
from xgboost import XGBClassifier
estimator = XGBClassifier(seed=40)

parameters = {
    'max_depth': range (2, 10, 2),
    'n_estimators': range(60, 240, 40),
    'learning_rate': [0.1, 0.01, 0.05]
}
grid_search = GridSearchCV(
    estimator=estimator,
    param_grid=parameters,
    scoring = 'recall',
    
    cv = 5,
    verbose=0
)

model=grid_search
In [91]:
#rf_shap_values = shap.KernelExplainer(grid_search.predict,dataset_aif_tranf_age.features)

2.a) with age as protected variable

In [92]:
mdl_age = model.fit(dataset_aif_tranf_age.features, dataset_aif_tranf_age.labels.ravel())

2.a.1) Model Explainability

2.a.1) Using SHAP

In [93]:
# explain the model's predictions using SHAP
# (same syntax works for LightGBM, CatBoost, scikit-learn, transformers, Spark, etc.)
explainer = shap.TreeExplainer(grid_search.best_estimator_,dataset_aif_tranf_age.features)
shap_values=explainer.shap_values(dataset_aif_tranf_age.features, dataset_aif_tranf_age.labels.ravel())
#https://github.com/slundberg/shap
In [94]:
shap_values
Out[94]:
array([[ 0.        ,  0.11156371,  0.00077599, ...,  0.00833379,
         0.        , -0.10627968],
       [ 0.        ,  0.07316086,  0.00077599, ...,  0.00833379,
         0.        ,  0.04096031],
       [ 0.        ,  0.06495592,  0.00077599, ...,  0.00833379,
         0.        ,  0.02756702],
       ...,
       [ 0.        , -0.18114036, -0.00346608, ..., -0.15743269,
         0.        ,  0.0271022 ],
       [ 0.        ,  0.0735417 ,  0.00315569, ...,  0.03174634,
         0.        ,  0.02756702],
       [ 0.        ,  0.07915238, -0.00067252, ...,  0.00833379,
         0.        ,  0.04096031]])
In [95]:
shap.initjs()
shap.force_plot(explainer.expected_value,shap_values[0,:], dataset_aif_tranf_age.feature_names)
Out[95]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [96]:
shap.initjs()
shap.force_plot(explainer.expected_value,shap_values[1,:], dataset_aif_tranf_age.feature_names)
Out[96]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [97]:
shap.force_plot(explainer.expected_value, shap_values[:,:], X.iloc[:,:],feature_names = dataset_aif_tranf_age.feature_names)
Out[97]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [98]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value, shap_values[0,:],feature_names=dataset_aif_tranf_age.feature_names)

2.a.2) Using ELI5

In [99]:
perm_age = PermutationImportance(model).fit(dataset_aif_tranf_age.features, dataset_aif_tranf_age.labels.ravel())
perm_imp=eli5.show_weights(perm_age,feature_names = dataset_aif_tranf_age.feature_names)
perm_imp
plt.show()
Out[99]:
Weight Feature
0.0224 ± 0.0026 NumMonths
0.0212 ± 0.0055 Savings_<500
0.0073 ± 0.0092 Age
0.0004 ± 0.0016 OtherPayBackPlan_bank
0 ± 0.0000 Foreignworker
0 ± 0.0000 Debtors_none
0 ± 0.0000 EmployDuration_>=7 yr
0 ± 0.0000 Purpose_others
0 ± 0.0000 Collateral_unknown/none
0 ± 0.0000 Savings_500+
0 ± 0.0000 Marital_Status
0 ± 0.0000 Gender
-0.0069 ± 0.0020 CreditAmount

2.b) gender as protected variable

2.b.1) Model Explainability

2.b.1) Using SHAP

In [100]:
mdl_gender = model.fit(dataset_aif_tranf_gender.features, dataset_aif_tranf_gender.labels.ravel())
In [101]:
# explain the model's predictions using SHAP
# (same syntax works for LightGBM, CatBoost, scikit-learn, transformers, Spark, etc.)
explainer = shap.TreeExplainer(grid_search.best_estimator_,dataset_aif_tranf_gender.features)
shap_values=explainer.shap_values(dataset_aif_tranf_gender.features, dataset_aif_tranf_gender.labels.ravel())
#https://github.com/slundberg/shap
In [102]:
shap_values
Out[102]:
array([[ 0.        ,  0.11156371,  0.00077599, ...,  0.00833379,
         0.        , -0.10627968],
       [ 0.        ,  0.07316086,  0.00077599, ...,  0.00833379,
         0.        ,  0.04096031],
       [ 0.        ,  0.06495592,  0.00077599, ...,  0.00833379,
         0.        ,  0.02756702],
       ...,
       [ 0.        , -0.18114036, -0.00346608, ..., -0.15743269,
         0.        ,  0.0271022 ],
       [ 0.        ,  0.0735417 ,  0.00315569, ...,  0.03174634,
         0.        ,  0.02756702],
       [ 0.        ,  0.07915238, -0.00067252, ...,  0.00833379,
         0.        ,  0.04096031]])
In [103]:
shap.initjs()
shap.force_plot(explainer.expected_value,shap_values[0,:], dataset_aif_tranf_gender.feature_names)
Out[103]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [104]:
shap.initjs()
shap.force_plot(explainer.expected_value,shap_values[1,:], dataset_aif_tranf_gender.feature_names)
Out[104]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [105]:
shap.force_plot(explainer.expected_value, shap_values[:,:], X.iloc[:,:],feature_names = dataset_aif_tranf_gender.feature_names)
Out[105]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [106]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value, shap_values[0,:],feature_names=dataset_aif_tranf_gender.feature_names)

2.b.2) Using ELI5

In [107]:
perm_gender = PermutationImportance(model).fit(dataset_aif_tranf_gender.features, dataset_aif_tranf_gender.labels.ravel())
perm_imp=eli5.show_weights(perm_gender,feature_names = dataset_aif_tranf_gender.feature_names)
perm_imp
plt.show()
Out[107]:
Weight Feature
0.0200 ± 0.0054 Savings_<500
0.0171 ± 0.0099 NumMonths
0.0086 ± 0.0075 Age
0 ± 0.0000 Foreignworker
0 ± 0.0000 OtherPayBackPlan_bank
0 ± 0.0000 Debtors_none
0 ± 0.0000 EmployDuration_>=7 yr
0 ± 0.0000 Purpose_others
0 ± 0.0000 Collateral_unknown/none
0 ± 0.0000 Savings_500+
0 ± 0.0000 Marital_Status
0 ± 0.0000 Gender
-0.0061 ± 0.0026 CreditAmount
In [ ]:
 

2.c) with marital status as protected variable

In [108]:
mdl_marital = model.fit(dataset_aif_tranf_marital.features, dataset_aif_tranf_marital.labels.ravel())

2.c.1) Model Explainability

2.c.1) Using SHAP

In [109]:
# explain the model's predictions using SHAP
# (same syntax works for LightGBM, CatBoost, scikit-learn, transformers, Spark, etc.)
explainer = shap.TreeExplainer(grid_search.best_estimator_,dataset_aif_tranf_marital.features)
shap_values=explainer.shap_values(dataset_aif_tranf_marital.features, dataset_aif_tranf_marital.labels.ravel())
#https://github.com/slundberg/shap
In [110]:
shap_values
Out[110]:
array([[ 0.        ,  0.11156371,  0.00077599, ...,  0.00833379,
         0.        , -0.10627968],
       [ 0.        ,  0.07316086,  0.00077599, ...,  0.00833379,
         0.        ,  0.04096031],
       [ 0.        ,  0.06495592,  0.00077599, ...,  0.00833379,
         0.        ,  0.02756702],
       ...,
       [ 0.        , -0.18114036, -0.00346608, ..., -0.15743269,
         0.        ,  0.0271022 ],
       [ 0.        ,  0.0735417 ,  0.00315569, ...,  0.03174634,
         0.        ,  0.02756702],
       [ 0.        ,  0.07915238, -0.00067252, ...,  0.00833379,
         0.        ,  0.04096031]])
In [111]:
shap.initjs()
shap.force_plot(explainer.expected_value,shap_values[0,:], dataset_aif_tranf_marital.feature_names)
Out[111]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [112]:
shap.initjs()
shap.force_plot(explainer.expected_value,shap_values[1,:], dataset_aif_tranf_marital.feature_names)
Out[112]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [113]:
shap.force_plot(explainer.expected_value, shap_values[:,:], X.iloc[:,:],feature_names = dataset_aif_tranf_marital.feature_names)
Out[113]:
Visualization omitted, Javascript library not loaded!
Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.
In [114]:
shap.plots._waterfall.waterfall_legacy(explainer.expected_value, shap_values[0,:],feature_names=dataset_aif_tranf_marital.feature_names)

2.c.2) Using ELI5

In [115]:
perm_marital = PermutationImportance(model).fit(dataset_aif_tranf_marital.features, dataset_aif_tranf_marital.labels.ravel())
perm_imp=eli5.show_weights(perm_marital,feature_names = dataset_aif_tranf_marital.feature_names)
perm_imp
plt.show()
Out[115]:
Weight Feature
0.0176 ± 0.0055 Savings_<500
0.0106 ± 0.0098 NumMonths
0.0102 ± 0.0082 Age
0 ± 0.0000 Foreignworker
0 ± 0.0000 OtherPayBackPlan_bank
0 ± 0.0000 Debtors_none
0 ± 0.0000 EmployDuration_>=7 yr
0 ± 0.0000 Purpose_others
0 ± 0.0000 Collateral_unknown/none
0 ± 0.0000 Savings_500+
0 ± 0.0000 Marital_Status
0 ± 0.0000 Gender
-0.0041 ± 0.0026 CreditAmount
In [ ]: